语义细分是农业机器人了解自然果园周围环境的一项基本任务。 LIDAR技术的最新发展使机器人能够在非结构化果园中获得准确的范围测量。与RGB图像相比，3D点云具有几何特性。通过将LIDAR和相机组合在一起，可以获得有关几何和纹理的丰富信息。在这项工作中，我们提出了一种基于深度学习的分割方法，以对来自激光镜像相机视觉传感器的融合数据进行准确的语义分割。在这项工作中探索和解决了两个关键问题。第一个是如何有效地从多传感器数据中融合纹理和几何特征。第二个是如何在严重失衡类条件下有效训练3D分割网络的方法。此外，详细介绍了果园中3D分割的实现，包括LiDAR-CAMERA数据融合，数据收集和标签，网络培训和模型推断。在实验中，我们在处理从苹果园获得的高度非结构化和嘈杂的点云时，全面分析了网络设置。总体而言，我们提出的方法在高分辨率点云（100k-200k点）上的水果分割时达到了86.2％MIOU。实验结果表明，所提出的方法可以在真实的果园环境中进行准确的分割。

与传统的刚性机器人相比，由于合规性，安全性和低成本，软机器人由于其优点而引起了越来越多的关注。作为软机器人的重要组成部分，软机器人夹具还显示出其优越的同时抓住具有不规则形状的物体。已经进行了最近的研究，以通过调整可变有效长度（VEL）来改善其抓握性能。然而，通过多室设计或可调刚度形状记忆材料实现的Vel需要复杂的气动电路设计或耗时的相变过程。这项工作提出了一种由3D印刷灯丝，忍者克朗的折叠式软机器人执行器。它是通过高速模型进行实验测试和表示的。进行数学和有限元建模，以研究所提出的软致动器的弯曲行为。此外，提出了一种拮抗约束机制来实现VEL，并且实验表明实现了更好的符合性。最后，设计了一种双模夹具，以展示Vel对抓取性能的进步。

现场机器人收获是农业产业近期发展的有希望的技术。在自然果园收获之前，机器人识别和本地化水果至关重要。然而，果园中收获机器人的工作空间很复杂：许多水果被分支和叶子堵塞。在执行操纵之前，估计每个果实的适当抓握姿势是很重要的。在本研究中，建议使用来自RGB-D相机的颜色和几何感官数据来执行端到端实例分段和掌握估计的几何意识网络A3N。此外，应用了工作区几何建模以帮助机器人操纵。此外，我们实施全球到本地扫描策略，它使机器人能够在具有两个消费级RGB-D相机中准确地识别和检索现场环境中的水果。我们还全面评估了所提出的网络的准确性和鲁棒性。实验结果表明，A3N达到了0.873的实例分割精度，平均计算时间为35毫秒。掌握估计的平均准确性分别为0.61厘米，4.8美元，中心和方向分别为4.8美元。总的来说，利用全球到局部扫描和A3N的机器人系统实现了从现场收集实验中的70 \％-85 \％的收获量的成功率。

桥梁采样是一种强大的蒙特卡洛方法，用于估计标准化常数的比率。引入了各种方法以提高其效率。这些方法旨在通过对它们应用适当的转换而不更改标准化常数来增加密度之间的重叠。在本文中，我们首先给出了最佳桥梁估计器的渐近相对平方误差（RMSE）的新估计器，通过等效地估计两个密度之间的$ f $差异。然后，我们利用此框架，并根据二元式转换提出$ f $ -gan桥估计器（$ f $ -GB），该框架将一个密度映射到另一个密度，并最小化最佳桥梁估计器的渐近RMSE相对于密度。通过使用$ f $ gan之间的密度之间的特定$ f $ divergence来选择这种转换。从某种意义上说，在任何给定的候选转换中，$ f $ -GB估算器可以渐近地实现比桥梁估算器低于或等于由任何其他转换的密度低的RMSE，我们显示出$ f $ -GB是最佳的。数值实验表明，$ f $ -GB在模拟和现实世界中的现有方法优于现有方法。此外，我们讨论了桥梁估计器如何自然来自$ f $ divergence估计的问题。

Temporal sentence grounding (TSG) aims to identify the temporal boundary of a specific segment from an untrimmed video by a sentence query. All existing works first utilize a sparse sampling strategy to extract a fixed number of video frames and then conduct multi-modal interactions with query sentence for reasoning. However, we argue that these methods have overlooked two indispensable issues: 1) Boundary-bias: The annotated target segment generally refers to two specific frames as corresponding start and end timestamps. The video downsampling process may lose these two frames and take the adjacent irrelevant frames as new boundaries. 2) Reasoning-bias: Such incorrect new boundary frames also lead to the reasoning bias during frame-query interaction, reducing the generalization ability of model. To alleviate above limitations, in this paper, we propose a novel Siamese Sampling and Reasoning Network (SSRN) for TSG, which introduces a siamese sampling mechanism to generate additional contextual frames to enrich and refine the new boundaries. Specifically, a reasoning strategy is developed to learn the inter-relationship among these frames and generate soft labels on boundaries for more accurate frame-query reasoning. Such mechanism is also able to supplement the absent consecutive visual semantics to the sampled sparse frames for fine-grained activity understanding. Extensive experiments demonstrate the effectiveness of SSRN on three challenging datasets.

New architecture GPUs like A100 are now equipped with multi-instance GPU (MIG) technology, which allows the GPU to be partitioned into multiple small, isolated instances. This technology provides more flexibility for users to support both deep learning training and inference workloads, but efficiently utilizing it can still be challenging. The vision of this paper is to provide a more comprehensive and practical benchmark study for MIG in order to eliminate the need for tedious manual benchmarking and tuning efforts. To achieve this vision, the paper presents MIGPerf, an open-source tool that streamlines the benchmark study for MIG. Using MIGPerf, the authors conduct a series of experiments, including deep learning training and inference characterization on MIG, GPU sharing characterization, and framework compatibility with MIG. The results of these experiments provide new insights and guidance for users to effectively employ MIG, and lay the foundation for further research on the orchestration of hybrid training and inference workloads on MIGs. The code and results are released on https://github.com/MLSysOps/MIGProfiler. This work is still in progress and more results will be published soon.

The ability for an agent to continuously learn new skills without catastrophically forgetting existing knowledge is of critical importance for the development of generally intelligent agents. Most methods devised to address this problem depend heavily on well-defined task boundaries, and thus depend on human supervision. Our task-agnostic method, Self-Activating Neural Ensembles (SANE), uses a modular architecture designed to avoid catastrophic forgetting without making any such assumptions. At the beginning of each trajectory, a module in the SANE ensemble is activated to determine the agent's next policy. During training, new modules are created as needed and only activated modules are updated to ensure that unused modules remain unchanged. This system enables our method to retain and leverage old skills, while growing and learning new ones. We demonstrate our approach on visually rich procedurally generated environments.

Zero-Shot Learning has been a highlighted research topic in both vision and language areas. Recently, most existing methods adopt structured knowledge information to model explicit correlations among categories and use deep graph convolutional network to propagate information between different categories. However, it is difficult to add new categories to existing structured knowledge graph, and deep graph convolutional network suffers from over-smoothing problem. In this paper, we provide a new semantic enhanced knowledge graph that contains both expert knowledge and categories semantic correlation. Our semantic enhanced knowledge graph can further enhance the correlations among categories and make it easy to absorb new categories. To propagate information on the knowledge graph, we propose a novel Residual Graph Convolutional Network (ResGCN), which can effectively alleviate the problem of over-smoothing. Experiments conducted on the widely used large-scale ImageNet-21K dataset and AWA2 dataset show the effectiveness of our method, and establish a new state-of-the-art on zero-shot learning. Moreover, our results on the large-scale ImageNet-21K with various feature extraction networks show that our method has better generalization and robustness.

To improve uncertainty quantification of variance networks, we propose a novel tree-structured local neural network model that partitions the feature space into multiple regions based on uncertainty heterogeneity. A tree is built upon giving the training data, whose leaf nodes represent different regions where region-specific neural networks are trained to predict both the mean and the variance for quantifying uncertainty. The proposed Uncertainty-Splitting Neural Regression Tree (USNRT) employs novel splitting criteria. At each node, a neural network is trained on the full data first, and a statistical test for the residuals is conducted to find the best split, corresponding to the two sub-regions with the most significant uncertainty heterogeneity. USNRT is computationally friendly because very few leaf nodes are sufficient and pruning is unnecessary. On extensive UCI datasets, in terms of both calibration and sharpness, USNRT shows superior performance compared to some recent popular methods for variance prediction, including vanilla variance network, deep ensemble, dropout-based methods, tree-based models, etc. Through comprehensive visualization and analysis, we uncover how USNRT works and show its merits.

Task-oriented dialog(TOD) aims to assist users in achieving specific goals through multi-turn conversation. Recently, good results have been obtained based on large pre-trained models. However, the labeled-data scarcity hinders the efficient development of TOD systems at scale. In this work, we constructed a weakly supervised dataset based on a teacher/student paradigm that leverages a large collection of unlabelled dialogues. Furthermore, we built a modular dialogue system and integrated coarse-to-fine grained classification for user intent detection. Experiments show that our method can reach the dialog goal with a higher success rate and generate more coherent responses.